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Abstract 

This study examined the cognitive levels of questions used by Iranian EFL teachers in 
advanced reading comprehension tests. Twenty teachers participated in this study and 
generated 215 questions which were then categorized according to Bloom's taxonomy. This 
taxonomy consists of six major categories which starts from the simplest behavior to the 
most complex. The lowest three levels are: knowledge, comprehension, and application. 
The highest three levels are: analysis, synthesis and evaluation. The results of this study 
showed that the most dominant question type was the 'knowledge' (54.21 %) followed by 
comprehension' questions (38.74 %). This indicated that, 92.43% of questions aimed at the 
first two levels of the taxonomy. Next to these two levels, the teachers' questions aimed at 
'synthesis' (2.33%), 'application' (1.86 %), 'evaluation' (1.39%) and 'analysis' (0.47 %) 
level, respectively. According to this analysis only 4.19% Iranian EFL teachers-generated 
questions were directed toward the highest three levels of Bloom's taxonomy, and 95.81% 
questions were aimed at the three lowest levels of Bloom's taxonomy. So, this study 
indicated that Iranian EFL teachers were aiming their teaching and testing primarily at the 
lowest cognitive levels. The importance of developing higher level thinking skills was 
discussed and suggestions were offered. 

Key words: Cognitive levels, high level questions, low level questions, critical thinking, 
critical readings 
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Introduction 



Teachers make use of different tools to initiate interaction in the classroom and research 
shows that the most common one is asking questions (Long and Sato 1983, Johnson 1990 
cited in Ellis 1994). Teacher-initiated questions enhance student learning by developing 
critical thinking skills, reinforcing student understanding, correcting student 
misunderstanding, providing feedback for students, and enlivening class discussion (Caram 
and Davis 2005). 

The focus of this study is on the cognitive levels of questions and the degree to which 
teachers use questions at different levels to foster critical thinking and reading skills. A 
further goal is to provide EEL teachers with a means to evaluate the ways in which the 
development of critical thinking and reading is facilitated in advanced reading courses. 
Questions were classified based on Bloom's Taxonomy. This study, therefore, seeks 
answers to the following questions. 

1. What cognitive levels of text processing are indicated by questions generated by 
EEE teachers in advanced reading texts? 

2. Which level of questions is emphasized more by teachers? High or low cognitive 
level of questions? 

Effective Uses of Questioning 

Through asking questions teachers help learners to develop lower and higher skills and to 
use their background knowledge to interact with the text. To find how to help students 
think critically, Unrau (2000 cited in Godfrey 2001) states that teacher should use 
questioning strategies that encourage students to engage in analysis, problem solving and 
inquiry. Questions function in both reading and teaching situations (Melnic 1968). In 



2 




reading, questions establish a basis for identifying and clarifying writers' purposes, which 
influences the method of reading, degree of comprehension, reading rate and skills 
employed. Considering the crucial role of questioning in both teaching and reading, it 
seems reasonable to expect teachers to make a conscious effort to make effective questions 
that help students to accomplish higher order thinking skills. 

Questioning Based on the Taxonomy of Educational Objectives 

Teacher-initiated questioning is a simple, yet strong method for exploring ideas and 
concepts. Testing for critical thinking involves advanced preparation. Benjamin Bloom and 
his colleagues (1956) presented their taxonomy of educational objectives as a basis for 
planning educational objectives, teaching-learning activities and assessment items. 
Taxonomy of educational objectives can help teachers to think more precisely about what 
they test on their exams. Bloom's taxonomy helps teachers clarify their intentions in 
teaching and testing, make their tests more challenging by teaching and testing to higher 
levels and provides a basis to asses student performance at ah of these levels, further it is a 
useful tool for planning critical reading in EFL classes (Surjosuseno and Watts 1999). 
Bloom's cognitive domain consists of six levels of knowledge, comprehension, application, 
analysis, synthesis, and evaluation from lower order to higher order thinking: 

Knowledge: Knowledge is defined as the remembering of previously learned material. 
Knowledge represents the lowest level of learning outcome in the cognitive domain. In 
EFL reading comprehension classes, knowledge questions are often used during and after 
reading a passage to encourage EFL classes to recall the content of the passage. 
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Comprehension: Comprehension is defined as the ability to grasp the meaning of the text. 
As Surjosuseno and Watts (1999) mentioned in EFL classes, there are three types of 
comprehension behavior: translation (translate from the second language to the first 
language), interpretation (explaining or summarizing) and extrapolation ( making 
predictions based on what is given in the passage as opposed to abstraction which is 
derived from experiences) . These learning outcomes go one step beyond the simple recall 
of information. Critical reading questions which require students to translate a passage are 
not relevant in EFL classes since both teachers and learners use the target language, but 
EFL learners need to interpret and extrapolate meaning during and after reading. 

Application: Application refers to the ability to use learned material such as rules, 
methods, concepts, principles, laws and theories in new and concrete situations. A critical 
reading teacher in EFL classes will ask application questions about the topic before, during 
and after reading a passage. Questioning before a reading encourages students to anticipate 
what is possible; questioning during the reading helps the learners to focus on the function 
of the topic and questioning after the reading helps the learners to apply the concept in a 
new situation. 

Analysis: Analysis refers to the ability to breakdown material into its component parts so 
that its organizational structure may be understood .This may include the identification of 
parts, analysis of the relationship between parts, and recognition of organizational 
principles. Learning outcomes here represent a higher intellectual level than comprehension 
and application because learners are required to understand both the content and structural 
forms of the material. Examples of learning objectives at this level are: recognize unstated 
assumption, recognize logical fallacies in reasoning, distinguish between facts and 
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inferences, evaluate the relevancy of data and analyze the organizational structure of a 
work (art, music, and writing). In critical reading for an EFL class analysis questions can 
be used during and after reading activities to encourage learners to understand the content 
and structure of the given passage. 

Synthesis: Synthesis refers to the ability to put parts together to form a new whole. 

Synthesis encourages students to create something new and rely on original and creative 
thinking. Students may make predictions and solve problems and offer different creative 
answers. Synthesis activities in an EFL class can include: solving problems which are 
described in the passage, and communicating with the author in the target language. 

Evaluation: Evaluation refers to the ability to judge the value of material, the solution to 
problem or the facts about particular cultures. Learning outcomes are highest in the 
cognitive hierarchy because they contain elements of all the other categories plus conscious 
value judgments. Critical reading in an EFL class may use evaluation as a means of 
focusing on learners' personal judgment derived from their existing schemata. 

Mainly there are two cognitive levels of questions: High and low cognitive level questions. 
Higher level questions are defined as those which ask students to mentally manipulate 
information previously learned to create an answer or to support an answer with logically 
reasoned evidence. Higher cognitive questions are also called open-ended, interpretive, 
evaluative, inquiry; inferential and synthesis questions (Cotton 1988). High level questions 
are those requiring complex application, analysis, evaluation or synthesis skills. Lower 
cognitive level questions are those which ask the students merely to recall actual words 
exactly in his / her own words material previously read or taught by teachers. They are 
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referred to in the literature as fact, closed, direct, recall and knowledge questions (Cotton 
1988). Low level questions are those at remembering, understanding, and lower level 
application. Tarlinton (2003) states that questions at higher level are most appropriate for: 

• Encouraging students to think more deeply and critically 

• Problem solving 

• Encouraging decision 

• Stimulating students to seek information on their own. 



and questions at the lower levels are appropriate for: 

• Evaluating students' preparation and comprehension 

• Diagnosing students' strengths and weaknesses 

• Revising or summarizing contents 

Critical Thinking and Critical Reading 

Every educational objective includes the goals of critical thinking and problem solving 
through the development of analytical and evaluative skills (Hamblen 1984). Critical 
reading refers to a careful, active, reflective, analytic reading. Critical thinking involves 
reflecting on the validity of what you have read in light of our prior knowledge and 
understanding of the world. Critical reading would appear to come before critical thinking: 
Only once we have fully understood a text (critical reading) can we truly evaluate its 
assertions (critical thinking). In actual practice, critical reading and critical thinking work 
together. Critical thinking allows us to monitor our understanding as we read. In other 
words critical thinking depends on critical reading. 
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Critical thinking implies that a reader is actively and constructively engaged in the process 
of reading. The reader is continually negotiating what s/he knows with what s/he is trying 
to make sense of the role of background knowledge and the student's ability to draw upon it 
is essential to critical thinking/learning. (Cooke, Dean 1970). Correia (2006) mentioned 
that the major benefit of the lesson on critical reading was the high level of enthusiastic 
student participation. This was attributable to some extent to the novelty of the task they 
performed: coming up with questions they thought would be answered in the text, rather 
than trying to answer superficial comprehension questions or true-false statements prepared 
by the textbook writer. Although developing critical reading skills can be time-consuming 
and difficult for EFL students, it can be accomplished with practice, and it is worth the 
effort 

Bloom's Taxonomy and Critical Reading 

The topic of teaching students to think while reading -critical reading- should be central to 
any discussion of thinking skills. Critical reading has been defined as learning to evaluate, 
draw inferences and arrive at conclusions based on the evidence (Zintz and Maggart, 1984 
cited in Carr 1990). Critical reading and thought-provoking exercises guide students 
through the process of developing the skills they need to become better readers. Although a 
variety of cognitive processes are suggested by various authors, most of the processes are 
similar in essence to the various levels of cognitive demand suggested by Bloom et al. Here 
are definitions of cognitive processes in critical reading as they were cited in Surjosuseno 
and Watt (1999). 
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Author 


Critical readers should be able to: 


Paul 


Participate in an inner dialogue with the writer and consider the 


(1993) 


writer's point of view by looking for key assumptions, major concepts, 
justifications, supporting examples, parallel experiences, implications 
and consequences, and means to interpret the text's meaning and 
assess it accurately and fairly. 


Flynn 


Analyze, synthesize and evaluate ideas through cooperative problem 


(1989) 


solving. 


Cheek et 


Know, comprehend, think, apply, analyze, syntheses, evaluate, relate 


al. 


information in the text to personal past experiences, interpret 


(1989) 


figurative language, determine the authors' purposes, evaluate the 
ideas presented, and apply the ideas presented to actual situations they 
have experienced. 


Hickey 


Suspend judgment until relevant facts are collected, and be willing to 


(1988) 


consider the author's viewpoint and allow for the possibility of bias. 


Rubin 


Collect, interpret, apply, analyze, and synthesize information. 


(1982) 


differentiate between fact and opinion, fantasy and reality, and be able 
to identify propaganda in written text. 



Considering these five definitions and Bloom's taxonomy, we ean understand most 
proeesses are similar with Bloom's cognitive demands. Following table can show this: 
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Similarities and differences between the five definitions of critical reading and Bloom's Cognitive Domain 





Paul 

(1993) 


Flynn 

(1989) 


Cheek 
et al. 
(1969) 


Hickey 

(1988) 


Rubin 

(1982) 


Knowledge 


+ 


- 


+ 


- 


+ 


Comprehension 


+ 


- 


+ 


- 


+ 


Application 


+ 


- 


+ 


- 


+ 


Analysis 


+ 


+ 


+ 


+ 


+ 


Synthesis 


+ 


+ 


+ 


- 


+ 


Evaluation 


+ 


+ 


+ 


+ 


+ 


+ = similarity; - = 


difference) 









Bloom's Taxonomy is used in various forms by advocates of critical reading. The 
Taxonomy correctly highlights the complexity of critical thinking (and critical reading) 
processes and provides a framework which encourages EFL teachers to plan a variety of 
learning activities which encourage critical reading. 

According to Surjosuseno and Watt (1999) it is reasonable to conclude that all six 
processes of Bloom's taxonomy are useful in developing learners' critical reading abilities 
in EFL since analysis, synthesis and evaluation processes are founded on knowledge, 
comprehension and application processes and each type of process is interdependent in 
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relation to the others. Thus Bloom's Taxonomy, when modified to suit the needs of a 
particular context, can be particularly useful as a tool for planning to teach critical reading 
in EFL classes. 

Method 

Participants 

This study was conducted with 20 EFL male and female teachers who have been teaching 
reading comprehension to advanced learners at different language institutes of Shiraz-Iran 
for more than five years. The reason for selecting these teachers is that they are 
experienced, and it is believed that these teachers would generate effective questions. 
Teachers are selected from different institutes randomly. 

Instrument 

The scoring system for the cognitive level of teachers' questions was based on Bloom's 
taxonomy .This taxonomy includes the range of cognitive processing from simple or low 
levels to higher levels like analysis, synthesis, and evaluation. It is one of the most 
universally applied models that provides a measure to organize thinking skills into six 
levels from the most basic to the higher levels of thinking, and it is a means of expressing 
qualitatively different kinds of thinking. 

Data collection 

The data collection process was done through collecting a sample of 215 questions 
generated by 20 teachers based on two advanced reading comprehension texts. After the 
required data had been collected; the questions of each test were categorized based on 
Bloom's taxonomy. While categorizing the questions, there were cases of overlap between 
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the categories identified. In these situations, such questions were placed in categories 
where it seemed to fall into most dominantly. Furthermore, two other colleagues who made 
themselves fa mi liar with Bloom's taxonomy reached a consensus with each other and the 
researcher. Frequency was used to determine the number of questions used by each teacher 
at each category; however the percentage more greatly demonstrated a difference in the 
level of questions used by each teacher. By finding the percentages of total questions in 
each category the following would be determined: 

1. Cognitive level of questions generated by teachers in advanced reading 
comprehension tests. 

2. Which levels of questions dominated more and which ones were used less 
frequently? 

3. To what extent teachers emphasized each level. 

All questions were counted despite the fact that some were repetitious 

Data analysis and Result 

From the total of 215 questions generated by teachers 123 embodied "knowledge”, 79 
"comprehension", 4 "application' which were considered as low level questions. Only 1 
question manifested analysis, 5 questions "synthesis' and 3 questions "evaluation", (see the 
Appendix 1) 

As the total number of questions varied from teacher to teacher, the percentage more 
greatly demonstrate a difference in the levels of questions used by each teacher .Table 1 
shows the percentage of questions at each level. 
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Table 1 



level 


Percentage 


knowledge 


57.21% 


comprehension 


36.74 % 


application 


1.86 % 


analysis 


0.47 % 


synthesis 


2.33 % 


evaluation 


1.39 % 



Considering the data in this table we can answer the research questions re-represented 
below: 

Question 1: What cognitive levels of text processing are indicated by questions generated 
by EFL teachers in advanced reading texts? 

The data indicate that on the whole, there are examples from all cognitive levels of text 
processing, but the tendency is towards lower level cognitive processes. On the whole, the 
most dominant question type was the" knowledge" (57.21 %) which means, the answers to 
these questions were explicit in the text and students could retrieve/locate them from text. 
After "knowledge" questions were "comprehension" questions (36.74 %) that require 
students to grasp the meaning of the text. As the table shows, 93.95% of questions aimed at 
the first two levels of the taxonomy. Next to these two levels, the teachers' questions aimed 
at "synthesis" (2.33%), "application" (1.86 %), "evaluation" (1.39 %) and "analysis" (0.47 
%) level, respectively. 

Question 2: Which level of questions is emphasized most by teachers, high or low 
cognitive level of questions? 
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Regarding high and low cognitive aspect, low level questions were emphasized by teachers 
as 95.81% questions were at low level and only 4.19 % questions were at high level. 
Table 2 shows this. 



Table 2 



Level 


Percentage 


Low 


95.81 


High 


4.19 



This table also shows that EFL advanced teachers prefer to use low level rather than high 
level questions in their tests. 

Discussion 

In general the figures above suggest that teachers demand little deep text processing. A few 
teachers ask students to explain their answers in ways that are diagnostic of deep 
comprehension. Almost all of the questions were questions on details. It is not that these 
questions do not somehow aid in comprehension but that a lot of these isolated questions to 
details stimulate and encourage students to read for details not for main ideas or real and 
deep comprehension. Possible explanations for teachers' preference for low level questions 
are that: 

1. Low level questions take a little time for the teacher to generate. Testing for critical 
thinking/reading involves advanced preparation. It is almost impossible to achieve critical 
thinking/reading if teachers write their tests the night before needed. Creating test items 
using Bloom's taxonomy requires time, however the effort is worthwhile. Writing and 
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evaluating levels of cognition in their tests i8s time consuming. As low level questions, 
especially knowledge questions, are easier to form, teachers prefer them. 

2. Teachers are mostly unaware of different cognitive levels of questions and learning. 
They may focus on one level and ignore others. According to the obtained results, it is not 
unusual to see teachers, who want their students to learn higher order thinking skills give 
examinations that require only lower level skills. It is also possible that teachers have not 
made themselves familiar with appropriate methods and framework for generating 
questions at high levels of cognition. 

3. Maybe teachers, themselves do not pay attention to higher level processes and do not 
expect their students to pay attention to these things 

4. Maybe our teachers consider knowledge and comprehension more important and focus 
on those things. 

Conclusion and suggestions 

The data from this study indicated that Iranian EFL teachers in advanced reading 
comprehension classes were aiming their testing primarily at the lowest cognitive levels of 
Bloom's taxonomy. Although developing critical reading skills may be time-consuming 
and difficult for EFL students, it can be accomplished with practice, and it is worth the 
effort. The taxonomy helps teachers clarify their intentions in teaching and testing and in 
creating materials that match their intentions. The taxonomy also allows teachers to make 
their test more challenging and testing to higher knowledge level. Teaching and testing in 
advanced reading comprehension is not to bombard students with low level questions 
containing hard vocabulary and complex grammatical point; rather it is to foster critical 
thinking and reading in students and to help EFL readers feel they have options in the way 
they choose to read the text and to help them feel in a more equal relationship with the 
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writer. All in all to improve the overall standard of education it is necessary that students 
think and operate at higher cognitive levels and their ability to do so is dependant on their 
teachers' ability to do the same. So, the findings of this study suggest that we need to 
improve questioning skills in teachers. The teacher educational programs need to take a 
careful look at this matter. Undergraduate method courses would be a good place to stress 
Bloom's taxonomy as students learn to use both high level and low level questions when 
they write lesson plans and tests. Johnson, Evans (1991) Reily, Joseph (1978) investigated 
the effect of question classification training on the cognitive level of questions. Their 
findings suggest that training has a significant effect on the cognitive level of questions 
used by teachers. Therefore teachers (experienced or novice) should be encouraged to 
attend seminars and teaching method classes which would be designed to enhance ability to 
reach higher cognitive levels in classroom discourse. 

I believe Bloom's taxonomy is the best guideline to decrease the gap between teachers' 
intended goals and the objectives they use in their tests. So teachers need to dedicate more 
time and effort to generate questions and use the taxonomy to ensure that they generate 
questions at the appropriate levels of cognition and that their intended objectives are met in 
their tests. 

Implication for educators 

The results of this study might assist evaluators to evaluate tests made by teachers and 
questions in course books more appropriately. Further this study might help teachers to 
design questions which are more effective. As a result of this study teachers can hopefully: 

• Use guidelines to study and evaluate questions they use in the classroom and tests 

• Develop an understanding of various questioning processes and thinking .skills. 
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• Develop skills in sequencing questions to facilitate students' meta- cognition and 
higher order cognition processing and skills. 

• Apply Bloom's theory of developing higher levels of thought process to advanced 
EFL classroom reading. 

• Assist their students not only to read the lines but to read between and beyond the 
lines. 

• Classify instructional objectives and goals 

The findings may also assist book-writers to design questions at different cognitive levels 
and select materials in the way to develop higher order thinking skills in students. 

This study may also help the researchers to develop an awareness of research in thinking, 
cognition, meta-cognition, and questioning. 
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Appendix 1 

Table: Frequency of teachers' questions 



Teachers 


Knowledge 


Comprehensi 

on 


Application 


Analysis 


Synthesis 


Evaluation 


Total 


T1 


2.00 


6.00 


.00 


.00 


1.00 


1.00 


10.00 


T2 




3.00 


.00 


.00 


.00 


.00 


10.00 


T3 




3.00 


.00 


.00 


.00 


.00 


10.00 


T4 




2.00 


.00 


1.00 


.00 


.00 


10.00 


T5 




4.00 


.00 


.00 


.00 


.00 


14.00 


T6 


4.00 


4.00 


1.00 


.00 


.00 


.00 


9.00 


T7 


5.00 


4.00 


.00 


.00 


.00 


.00 


9.00 


T8 


12.00 


3.00 


.00 


.00 


.00 


.00 


15.00 


T9 


10.00 


2.00 


.00 


.00 


.00 


.00 


12.00 


TIO 


8.00 


3.00 


.00 


.00 


.00 


.00 


11.00 


Til 


11.00 


3.00 


.00 


.00 


.00 


.00 


14.00 


T12 


3.00 


6.00 


1.00 


.00 


.00 


.00 


10.00 


T13 


8.00 


2.00 


.00 


.00 


.00 


.00 


10.00 


T14 


2.00 


4.00 


.00 


.00 


2.00 


2.00 


10.00 


T15 


3.00 


6.00 


1.00 


.00 


.00 


.00 


10.00 




6.00 


4.00 


.00 


.00 


.00 


.00 


10.00 




5.00 


5.00 


.00 


.00 


.00 


.00 


10.00 




1.00 


6.00 


.00 


.00 


1.00 


.00 


8.00 




10.00 


2.00 


1.00 


.00 


.00 


.00 


13.00 




2.00 


7.00 


.00 


.00 


1.00 


.00 


10.00 




123 


79 


4 


1 


5 


3 


215 
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